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Background: The value of integrated care through comprehensive, coordinated, and 
family-centered services has been increasingly recognized for improving health outcomes of 
children with special health care needs (CSHCN). In a randomized controlled trial (RCT), the 
integrated care provided through the Children's Treatment Network (CTN) was compared with 
usual care in improving the psychosocial health of target CSHCN. In this paper, we aimed to 
estimate the effect of CTN care by conducting multiple analyses to handle noncompliance in 
the trial. 

Methods: The trial recruited target children in Simcoe County and York Region, ON, Canada. 
Children were randomized to receive CTN or usual care and were followed for 2 years. The CTN 
group received integrated services through multiple providers to address their specific needs 
while the usual care group continued to receive care directed by their parents. The outcome was 
change in psychosocial quality of life at 2 years. We conducted intention-to-treat, as-treated, 
per-protocol, and instrumental variable analyses to analyze the outcome. 
Results: The trial randomized 445 children, with 229 in the intervention group and 216 in the 
control group. During follow-up, 52% of children in the intervention group did not receive 
complete CTN care for various reasons. At 2 years, we did not find a significant improvement 
in psychosocial quality of life among the children receiving CTN care compared with usual 
care (intention-to-treat mean difference 1 .50, 95% confidence interval -1 .49 to 4.50; P = 0.32). 
Other methods of analysis yielded similar results. 

Conclusion: Although the effect of CTN care was not significant, there was evidence showing 
benefits of integrated care for CSHCN. More RCTs are needed to demonstrate the magnitude of 
such an effect. The CTN study highlights the key challenges in RCTs when assessing interven- 
tions involving integrated care, and informs fiirther RCTs including similar evaluations. 
Keywords: children with special health care needs, chronically ill, family-centered care, 
randomized controlled trial, noncompliance 
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Introduction 

Children with special health care needs (CSHCN) are those "who have or are at 
increased risk for a chronic physical, developmental, behavioral, or emotional condi- 
tion, and who also require health and related services of a type or amount beyond that 
required by children generally".' In Canada, about 13%-18% of children are consid- 
ered to have special needs. ^ These children usually have poor quality of life^ associ- 
ated with emotional symptoms,'' psychologic problems,^ and health risk behaviors.'' 
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Two thirds of parents with a disabled child do not get the 
necessary treatments for their child and more than one third 
do not know where to look for help.' These parents endure 
enormous financial and mental burdens due to their child's 
illnesses.* Since most CSHCN live at home, parents have to 
quit their jobs, work fewer hours, turn down promotion, or 
compromise job performance' to take care of their child with 
special needs. "Finding services, battling waitlists, travel- 
ling long distances, briefing numerous professionals from 
different sectors on their child's history, and coordinating 
services from multiple agencies" are just part of their daily 
life.'" These parents commonly describe continuing strain 
and chronic fatigue concerning their child's well-being," and 
over 35% of them report high levels of emotional distress.'^ 
Their families are also affected by high divorce rates. 

CSHCN are usually diverse in nature and need an indi- 
vidualized approach to address their specialized and multiple 
needs.''' The value of continuous, comprehensive, coordi- 
nated and family-centered care'^ has been widely recognized. 
Delivery of this new type of care requires a more integrative 
approach than the traditional one through: collaboration with 
parents to decide on case-specific goals; coordination of an 
individual team of service providers; and provision of a single 
point of access to address families' multiple needs. As an 
example, the medical home"' in the US is a new development 
in health care delivery for families with CSHCN. It has led to 
reductions in wait-time and unmet health care needs for target 
children," increased use of preventive care,'* decreased risk of 
comorbidities," and alleviation of family burden.^" Adoption 
of these strategies in the Canadian system has been limited. 
There are examples of innovative interventions (for example, 
the chronic care model in British Columbia and the Family 
Health Network in Ontario), but these interventions target 
adult patients.'" Compared with adults, CSHCN are affected 
by a larger number of uncommon conditions and demand 
more tailored services and a more complex model of care. 

In 2006, a new model of care for CSHCN and their 
families, known as the Children's Treatment Network (CTN), 
was launched in Simcoe County and York Region in Ontario, 
Canada. The CTN model of care is funded by the government 
and is based on collaboration of local service providers from 
different agencies and organizations. It provides a single 
point of access and service navigation for children. The CTN 
model is unique, in that it assigns a service coordinator and 
an individual team of health service providers for each target 
family, develops a single plan of care, and uses an electronic 
record system to share clinical information and assessment 
of the child. Some services available to families through the 



CTN include developmentally appropriate therapy, speech 
therapy, augmentative communication, in-home social sup- 
port, behavior therapy, and early childhood education (www 
ctn-simcoeyork.ca ). Before the launch of the CTN, parents 
in both communities had limited access to the specialized 
treatments needed for their children and had to travel outside 
of the region to access those services. Funded by the Ministry 
of Children and Youth Services and the Ministry of Health 
and Long-term Care In Ontario, Canada, Thurston et aP' 
conducted a randomized controlled trial to compare the inte- 
grated interventional program of care organized through the 
CTN versus usual care for managing children with multiple 
and complex health needs in Simcoe County and York Region 
(trial registration in clinicaltrials.gov NCT0I379443). This 
CTN trial was subject to substantial noncompliance, where 
about 52% of children in the intervention group only received 
parts of CTN integrated care. Some analyses of the CTN were 
published previously.^^-^' 

The primary goal of the CTN trial was to assess the effect 
of CTN integrated care on improving the psychosocial func- 
tion of target children in Simcoe County and York Region. In 
this study, we compared the intention-to-treat (ITT) method 
with alternative methods of analyzing the data. We also aimed 
to assess the sensitivity of ITT analysis for participants' 
noncompliance with the CTN intervention. 

Materials and methods 

We describe the CTN trial here by adopting the Consolidated 
Standards of Reporting Trials guideline.^'' Using a parallel 
design, each eligible child was randomized to an intervention 
group or a control group with a 1 : 1 allocation ratio. 

Participants 

Participants were recruited from the Simcoe County and 
York Region in Ontario, Canada. The Simcoe York District 
Health CounciP' projected the total number of children in 
both regions to be 390,498 in 2007. According to a national 
survey, 1% of children were estimated to have severe dis- 
abilities and receive rehabilitation services.^' 

Children aged 0-19 years were recruited if they had any 
of the following conditions: cerebral palsy, brain injury, 
developmental difficulties, Down syndrome, spina bifida, 
autism, physical disability, developmental disability, perva- 
sive developmental disorder, or a chronic medical condition. 
Children who were receiving palliative care, requiring emer- 
gency services, or living outside of the region at the time 
were excluded. Non-English speaking families without an 
English translator were also excluded. 
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Potential children were identified through the Commu- 
nity Care Access Centres, school boards, and Early Interven- 
tion Services agencies. Eligible families were contacted by 
mail with an invitation package, which contained a study 
information letter, and a parental consent and participant 
contact information form. A second mail-out was done to 
families who did not respond to the initial mail-out. Only 
children whose parents had provided written informed con- 
sent could participate. Children were enrolled from May to 
December 2007. A unique patient ID was created for each 
enrolled child. 

Ethics 

There were no known harms or safety risks to the children 
and their families involved in the trial. The research eth- 
ics board at Hamilton Health Sciences/McMaster Health 
Sciences approved the study. 

Intervention 

Each child in the CTN group was assigned a service 
navigator who conducted a comprehensive assessment to 
identify the child's health conditions. A trained service 
coordinator then followed up with the family. In working 
with the parents, an individual team of service providers was 
formed according to the child's health and social needs. For 
example, a team comprising an augmentative communica- 
tion services consultant, early interventionist, occupational 
therapist, physiotherapist, speech and language therapist, 
and service coordinator was assembled for a child with 
cerebral palsy. This integrative team together with fam- 
ily members developed a single plan of care for the child. 
The service coordinator organized the delivery of services 
according to the plan. The team met with the family for ongo- 
ing assessment and revision of the plan on a regular basis. 
All assessments and clinical notes concerning the child were 
documented in an electronic record system shared by all team 
members. Figure 1 shows the detailed components of CTN 
integrated care. Families in the control group continued to 
manage services for their child in a self-directed manner. 
This group did not have access to the CTN integrated care; 
however, they had access to all other service providers 
(including CTN partners) as requested. 

Fidelity to integrated child and family team care was 
evaluated by: content in the child and family team of 
providers (extent, scope, and agreement in collaboration); 
quality of team functioning, measured by the Partnership 
Functioning Scale;^' and the frequency and duration cov- 
erage of the prescribed services as outlined in the single 



plan of care compared with providers' entries on electronic 
record. 

Outcome and variables 

The outcome was the change in the child's psychosocial qual- 
ity of life (QoL) in 24 months. This was reported by parents 
using the short form Pediatric Quality of Life (SF PedsQL).^^ 
Psychosocial QoL was the sum of the emotional, social, and 
school fiinctions and was extrapolated to have a score from 0 
(the worst QoL) to 100 (the best QoL). Information on each 
child's age, sex, and admission diagnosis was collected. The 
age, marital status, educational level, and family income of 
the parent who was the most knowledgeable (PMK) about 
the child were recorded. 

Parental distress score 

The Kessler scale (KIO)^' was used to measure PMK symp- 
toms of depression and anxiety. Ten questions measured 
these feelings: sad, nervous, restless, hopeless, worthless, 
everything was an effort, tired for no good reason, so nervous 
that nothing could calm down, so restless and could not sit 
still, and depression during the past month. The PMK rated 
each item on a five -point scale (1, "all of the time" to 5, 
"none of the time"). Scores range from 10 (no symptom of 
distress) to 50 (severe distress). 

Positive parenting score 

The National Longitudinal Survey of Children and Youth 
parenting scale was used, and five questions adapted from 
the Parent Practices Scale^" were used to measure positive 
parenting behaviors (eg, praise, play). The PMK rated each 
item (eg, "Do something special with your child that he/she 
enjoys") on a five-point scale (0, "never" to 4, "many times 
each day"). Higher scores indicate more positive parenting 
behaviors. 

Social support score 

An eight-item shortened version of the Social Provisions 
Scale^' was used to measure the level of social support that the 
PMK received in guidance, reliable alliance, and attachment. 
PMK rated each item on a four-point scale (0, "strongly 
disagree" to 3, "strongly agree"). Higher scores represent a 
greater level of social support. 

Family function score 

Thirteen items taken from the National Longitudinal Survey 
of Children and Youth population survey, based on a subscale 
of the McMaster Assessment Device,^^ were used to measure 
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CTN integrated care 
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Recruitment, baseline measures, 
randomization 




During 24 months 
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At 12 and 24 months 


Measurement of outcomes 



H 


Government funded the CTN: board established, executive and associate directors hired, integrated 
service agreements accomplished, meetings with network partners to monitor ongoing network 
development and address challenges 




Government funded to enhance rehab and mental health services throughout Simcoe County 
and York Region 


□ 


Single point of access: the CTN established the access form and hired workers called "system 
navigators", who referred information to the child and family coordinator 




The CTN established 10 local teams made up of interdisciplinary clinicians from participating 
partners' (1) training in creating a single plan of care' (2) training in integrated service team 
functioning; and (3) training team coordinators 




e 




The CTN set up and trained the use of the e-record 


m 


REB approved the RCT. Investigators pilot-tested recruitment, enrolled children, conducted baseline 
measures, and randomized children into either group 


0 


The CTN hired and trained team facilitators. A child and family team for each case was formed from members 
of local teams. The CTN assessed the integration of partners and the functioning of child and family teams 

• Single point of access: one number to call to access any service for child or family 

• System navigator: did intake; set up e-record; assessed problems; and identified regional team coordinators 

• Team coordinator: assembled team members and skills to match child and family needs; conducted team 
meetings; and arranged service sequence and timing 

• Child and family team: agreed on goals and single plan of care for child and requested services. 

• Electronic record: single point of access to obtain and record information; common record available to all 
team members to record notes and progress 




0 






0 




Usual waitlist as a function of prioritization 



I I = fixed objects O = flexible activities 



Figure I Components of the Children's Treatment Network versus usual care. 

Abbreviations: CTN, Children's Treatment Network; RCT, randomized controlled trial; REB, research ethics board. 



various aspects of family functioning in problem-solving, 
communication, affective responsiveness, affective involve- 
ment, and behavior control. The PMK rated each item on 
a four-point scale (0, "strongly agree" to 3, "strongly dis- 
agree"). Scores of negatively oriented items (eg, "We avoid 
discussing our fears or concerns") were reversed so that 
higher scores represent greater family dysfunction. 

Sample size 

Using a minimum clinically important difference of ten on 
the target children's psychosocial QoL" and an estimated 
standard deviation of 28 estimated from our pilot sample, 
the sample size was calculated to be 240 for a two-sided 



test with 5% alpha and 80% statistical power. To allow for 
ten independent variables and an attrition rate of 20%, the 
required sample size was 425. 

Randomization 

Using a parallel design, eligible children with parental 
consent were randomized to the intervention and control 
groups with a 1:1 allocation ratio. Children were strati- 
fied by region (Simcoe/York), Community Care Access 
Centre, and age (pre-school/school). They were random- 
ized within stratum by using a block size of six. The ran- 
domization list was generated by the Health and Social 
Service Utilization Research Unit (HSSURU) at McMaster 
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University ( flis.mcmaster.ca/slru ). The allocation codes 
were then sequentially linked to the patient IDs for enrolled 
children. 

Blinding 

Trained interviewers at the HSSURU, who were blinded to 
group allocation, collected data using a standardized ques- 
tionnaire booklet containing questions on demographics and 
outcome measures. Data collection was performed by tele- 
phone at baseline, and at 12 and 24 months. After enrollment, 
children remained anonymous and were identified by their 
patient IDs. Only the statistician responsible for randomiza- 
tion had access to the allocation codes. The data analyst was 
blinded to group allocation; however, the participants were 
aware of their allocation. All data were stored and maintained 
at the HSSURU. 

Statistical methods 

We adopted the ITT principle by analyzing all patients 
according to how they were randomized, regardless of 
whether they complied with the assigned treatment or not. 
The ITT estimate was obtained by fitting a linear regression 
model with adjustment for baseline variables. The multiple 
imputation technique was used to impute the missing data by 
assuming that they were missing at random. Five imputation 
datasets were produced and combined to produce an estimate 
of 98.5% relative efficiency, given a fraction of about 30% 
missing data.''* 

In the sensitivity analysis of the impact of noncompli- 
ance, we employed three other analytical approaches, ie, 
as-treated, per-protocol, and instrumental variable analyses. 
In the as-treated analysis, we compared the treated children 
(those who received complete CTN integrated care) with the 
untreated ones (those who received parts or none of the CTN 
integrated care) to estimate the effect of the intervention. In 
the per-protocol analysis, we analyzed the children by their 
randomization, but only ones who complied with the assigned 
treatment. Children in the control group did not have access to 
CTN integrated care and were deemed to comply with usual 
care. Both as-treated and per-protocol analyses compromised 
original randomization and were prone to selection bias and 
confounding. We employed the propensity score technique to 
adjust for those factors. Application of the propensity score 
in the as-treated and per-protocol analyses could potentially 
be a novel approach. Because we could not find any study 
comparing different uses of the propensity score in this 
context, we included all four commonly used propensity 
score methods, ie, matching, stratification, weighting, and 



covariate adjustment. The details of these propensity score 
methods are given in the Supplementary materials section. 

The instrumental variable analysis" estimated the effect 
of CTN integrated care through an instrumental variable. 
The randomization allocation (Z) was associated with the 
exposure of CTN integrated care (X) but independent of 
confounders and any factor that explained the outcome (Y). 
Thus, Z satisfied the conditions to be an instrumental variable, 
and the treatment effect can be estimated by 



Y) 



A 



(Z-X) 



where yS^^j^ is the association between randomization and the 
outcome and fi^^_^ is the association between randomization 
and the receipt of CTN integrated care. We assumed that the 
Z-Y association was independent of the Z-X association. 
Under the assumption, we used the delta method to derive 
the variance of /J^^^for calculating the 95% confidence interval 
(CI) and P- value. The details are provided in the Supplemen- 
tary materials section. 

We compared the ITT method with other methods of analy- 
sis to assess the sensitivity of the results. Figure 2 summarizes 
the different approaches to analysis. The results are expressed 
as the mean difference, 95% CI of the mean difference, and 
corresponding P-value. The level of significance was set at 
5%. All analyses were performed using statistical software R 
version 2.12.1 and Statistical Package for the Social Sciences 
version 19 software (IBM Corporation, Armonk, NY, USA). 



Statistical analysis 



Intention-to-treat 
(ITT) 



As-treated (AT) 



PS matching 



Stratification on PS 



Weighting on PS 



Covariate 
adjustment with PS 



Per-protocol (PP) 
methods 



Instrument variable 
(IV) 



r— PS matching 



Stratification on PS 



Weighting on PS 



Covariate 
adjustment with PS 



Figure 2 Summary of different analytical approaches. 
Abbreviation: PS, propensity score. 
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Results 

Recruitment, baseline data, 
and participant flow 

Between May and December 2007, 2,319 eligible children 
were identified and their families were contacted by 
mail. Parents of 465 eligible children returned signed 
consent forms to indicate their willingness to participate. 
Later, 20 consented parents did not complete the baseline 
interview and were excluded from the study. The remaining 
445 children were randomized to either the intervention group 
(n = 229) or the control group (n = 216). 

The children were followed until December 2009. During 
the 2-year follow-up, 53 children in the CTN group did not 
have a team of service providers assembled or did not have 
the services available for their specific needs. Another 58 
children in the CTN group withdrew from CTN integrated 
care. Common reasons for withdrawal were: death of the 
child, movement of the family outside of the region, a par- 
ent's decision to manage the child without assistance, and 
transfer to another health care center. Overall, 64 (28%) 
children in the intervention group and 57 (26%) children 
in the control group were lost to follow-up. Figure 3 shows 



the flow of participants in the study. All 445 children were 
analyzed using a multiple imputation technique. 

Statistical analysis 

The children's baseline characteristics are summarized in 
Table 1 . Categorical and continuous variables are expressed 
as the frequency (percentage) or mean (standard deviation), 
respectively. Within the CTN arm, baseline variables were 
compared between the children who received complete CTN 
integrated care (treated) and those who received partial or 
none of the CTN integrated care (untreated). The results 
of this comparison are reported in Table 2. From the com- 
parison, we did not find any significant differences between 
treated and untreated children except for parenting style 
(mean difference IA;P= 0.02); however, a mean difference 
of 1 .4 on a score ranging from 0 to 20 did not seem to be a 
clinically relevant association. 

We conducted different analyses to estimate the effect of 
CTN integrated care. The results did not show a significant 
improvement in the children's psychosocial QoL in the CTN 
group compared with the usual care group (mean difference 
1.50, 95% CI -1.49, 4.50; P = 0.32). The results for the 



Assessed for eligibility (n = 2,319) 



Excluded (n = 1,874): 
^ ♦No response; declined to participate 
♦ Did not complete baseline interview 



Randomization (n = 445) 



Allocated to intervention (n = 229) 

♦ Treated children (n = 1 1 1 ) 

♦ Untreated children (n = 118): 

1. Ongoing integration (n = 7) 

2. Coordinated but did not receive services (n ■■ 

3. Declined CTN services (n = 58) 



53) 



Lost to follow-up (n = 64): 

- 24 from the treated 

- 40 from the untreated 



Allocated to control (n = 216) 
♦ Received usual care (n = 216) 





Lost to follow-up 




(n = 57) 



Analysis: 

♦ Intention-to-treat analysis: (n = 229) 

♦ As-treated analysis: (n = 229: 111 treated ,118 untreated) 

♦ Per-protocol analysis: (n = 111) 

♦ Instrumental variable analysis: (n = 229) 



Analysis: 

♦ Intention-to-treat analysis: (n = 216) 

♦ As-treated analysis: (n = 216; all untreated) 

♦ Per-protocol analysis: (n = 216) 

♦ Instrumental variable analysis: (n = 216) 



Figure 3 Flow of children in the Children's Treatment Network trial. 
Abbreviation: CTN, Children's Treatment Network. 
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Table I Baseline characteristics 



Baseline variable 


Intervention 


Control 


P-value 




(n = 229) 


(n = 2 1 6) 




Child's age, mean (SD) 


7.8 (4.3) 


8.1 (4.6) 


0.60 


Child's sex, frequency (%) 








Male 


148 (64.6%) 


149 (69.0%) 


0.33 


Female 


81 (35.4%) 


67 (3 1 .0%) 




Child's admission diagnostic, 


frequency (%) 






Mental and developmental 


106 (46.3%) 


95 (44.0%) 


0.81 


disorders 








Diseases of the nervous 


S3 (23.1%) 


51 (23.6%) 




system 








Congenital abnormalities 


39 (17.0%) 


34 (15.7%) 




Other disease 


3! (13.5%) 


36 (16.7%) 




Child's psychosocial score. 


59.0 (18.6) 


59.2 (18.6) 


0.85 


mean (SD) 








Parent's age, mean (SD) 


40.5 (7.6) 


40.4 (7.7) 


0.95 


Marital status, frequency (%) 








Married including 


190 (83.0%) 


187 (86.6%) 


0.29 


common-law 








Other 


39 (17.0%) 


29 (13.4%) 




Parent's education, frequency (%) 






Secondary 


83 (36.4%) 


78 (36.1%) 


0.95 


Post-secondary 


145 (63.6%) 


138 (63.9%) 




Family annual income, frequency (%) 






<$30,000 


32 (14.0%) 


32 (14.9%) 


0.96 


$30,000-$90,000 


121 (53.1%) 


1 14 (53.0%) 




>$90,000 


75 (32.9%) 


69 (32.1%) 




Parent's Kessler distress 


19.5 (5.8) 


20.4 (7.2) 


0.16 


score, mean (SD) 








Positive parenting score. 


15.2 (3.1) 


15.1 (3.0) 


0.82 


mean (SD) 








Social support score. 


17.6 (4.7) 


17.5 (4.3) 


0.74 


mean (SD) 








Family functioning score. 


9.1 (6.3) 


9.4 (5.9) 


0.60 


mean (SD) 









Notes: Continuous and categorical variables are expressed as the mean (standard 
deviation) or frequency (percentage), respectively. The P-value was calculated based 
on a t-test for continuous variables and a chi-square test for categorical variables. 
Abbreviation: SD, standard deviation. 



estimated treatment effect are reported in Table 3 . In general, 
the conclusion from the ITT analysis was consistent with that 
drawn from the as-treated, per-protocol, and instrumental 
variable analyses. We also noticed some systematic patterns 
in the direction, magnitude, and precision of the estimates. 
All estimates except for the as-treated estimates favored CTN 
integrated care. The per-protocol estimates were the closest 
to 0 which represented no difference between CTN integrated 
care and usual care. Both ITT and instrumental variable 
estimates showed a larger effect of CTN integrated care over 
usual care than the per-protocol estimates. The instrumental 
variable estimate had the largest 95% CI. For the as-treated 
and per-protocol analyses which compromised the original 
randomization, adjusting for confounders and imbalance by 
matching on the propensity score produced estimates with a 



Table 2 Comparing treated with untreated children in the 
intervention group 



Baseline variable 


Treated 


Untreated 


P-value 




children 


children 






(n = III) 


(n = 1 1 8) 




Child's age, mean (SD) 


7.3 (4.3) 


8.3 (4.3) 


0.09 


Child's sex, frequency (%) 








Male 


70 (63.1%) 


78 (66.1%) 


0.63 


Female 


41 (36.9%) 


40 (33.9%) 




Child's diagnosis on admission. 


frequency (%) 






Mental and developmental 


46 (41.4%) 


60 (50.8%) 


0.13 


disorders 








Diseases of the nervous 


32 (28.8%) 


21 (17.8%) 




system 








Congenital abnormalities 


21 (18.9%) 


18 (15.3%) 




Other disease 


12 (10.8%) 


19 (16.1%) 




Child's psychosocial score. 


58.1 (19.0) 


59.9 (18.4) 


0.49 


mean (SD) 








Parent's age, mean (SD) 


40.4 (6.9) 


40.6 (8.3) 


0.81 


Marital status, frequency (%) 








Married including 


92 (82.9%) 


98 (83.1%) 


0.97 


common-law 








Other 


19 (17.1%) 


20 (16.9%) 




Parent's education, frequency (%) 






Secondary 


40 (36.0%) 


43 (36.8%) 


0.91 


Post-secondary 


71 (64.0%) 


74 (63.2%) 




Family annual income, frequency (%) 






<$30,000 


19 (17.3%) 


13 (1 1.0%) 


0.34 


$30,000-$90,000 


58 (52.7%) 


63 (53.4%) 




< $90,000 


33 (30.0%) 


42 (35.6%) 




Parent's Kessler distress 


20.2 (5.9) 


18.8 (5.6) 


0.09 


score, mean (SD) 








Positive parenting score. 


15.7 (3.2) 


14.8 (3.0) 


0.02 


mean (SD) 








Social support score. 


17.7 (4.6) 


17.6 (4.9) 


0.86 


mean (SD) 








Family functioning score. 


9.1 (6.4) 


9.1 (6.3) 


0.97 


mean (SD) 









Notes: Continuous and categorical variables are expressed as the mean (standard 
deviation) or frequency (percentage), respectively. The P-value was calculated based 
on a t-test for continuous variables and a chi-square test for categorical variables. 
Abbreviation: SD, standard deviation. 



wider 95% CI than did the other propensity score methods. 
Figure 4 shows a comparison of the estimates obtained from 
different analyses. 

Discussion 

The ITT analysis did not show a significant improvement in 
psychosocial QoL among children receiving CTN integrated 
care than those receiving usual care in Simcoe County and 
York Region over 2 years. We conducted as-treated, per- 
protocol, and instrumental variable analyses to assess the sen- 
sitivity of this conclusion under substantial noncompliance 
with the CTN intervention. These alternative analyses also 
showed no significant difference between groups. Previously, 
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Table 3 Summary of estimates of treatment effect 





Multiple 1 


mputation 






Raw data 










73 /o ^1 




D 1~ 

P-value 


Mn 






— n — ;~ 

P-V3.IUG 


ITT analysis' 


1 .50 


-1.49 


4.50 


0.32 










AT analysis 


















Matched by PS 


-2.60 


-7.45 


2.24 


0.29 


-1.82 


-6.49 


2.86 


0.45 


Stratified by quintiles of PS 


-0.89 


-4.34 


2.57 


0.61 


-1.17 


-5.0! 


2.68 


0.55 


Weighted by PS 


-0.75 


-3.75 


2.25 


0.62 


-0.80 


-4.10 


2.50 


0.63 


Adjusted by PS 


-1.12 


-4.62 


2.39 


0.53 


-1.15 


-4.99 


2.70 


0.56 


PP analysis 


















Matched by PS 


0.67 


-3.44 


4.78 


0.75 


2.26 


-2.60 


7.12 


0.36 


Stratified by quintiles of PS 


0.37 


-3.33 


4.06 


0.85 


0.47 


-3.58 


4.52 


0.82 


Weighted by PS 


0.21 


-3.24 


3.66 


0.91 


0.70 


-3.16 


4.56 


0.72 


Adjusted by PS 


0.02 


-3.23 


3.27 


0.99 


-0.20 


-3.7! 


3.31 


0.9! 


IV analysis 


3.10 


-3.08 


9.29 


0.33 


5.10 


-0.78 


10.97 


0.09 



Note: MTT analysis requires the inclusion of all randomized patients and was only performed when patients' missing observations were imputed by multiple imputation. 
Abbreviations: MD, mean difference {difference in change of psychosocial score over 2 years between groups); CI, confidence interval; ITT, intention-to-treat; 
AT, as-treated; PP, per-protocol; IV, instrumental variable; PS, propensity score. 



two CTN studies have been published,^^-^^ which explored 
the interactions between multiple factors and system inte- 
gration on the child's psychosocial QoL and examined the 
associations between multiple factors and level of psychiatric 
distress experienced by parents. 



Our findings were limited by a number of factors. First, 
the child's psychosocial QoL was reported by their parents. 
Parents might have limited knowledge concerning their chil- 
dren's health-related QoL. The parents' responses reflected 
their own perception of children's disease-related experiences, 



Multiple Imputation: 




AT-matched by PS 
AT-stratified by PS 

^ AT-weighted by PS 
AT-adjusted by PS 

-K PP-matched by PS 
PP-stratified by PS 
PP-weighted by PS 
PP-adjusted by PS 

Raw data: 

AT-matched by PS 
AT-stratified by PS 
AT-weighted by PS 
— ^ AT-adjusted by PS 
^ PP-matched by PS 
—A— PP-stratified by PS 
— PP-weighted by PS 
-■- PP-adjusted by PS 



-4 -2 0 2 4 6 

Mean difference with 95% confidence interval 



Figure 4 Comparing the estimates of treatment effect from different analyses. 

Note: Mean difference refers to the difference in the change in psychosocial score in 2 years between groups. 
Abbreviations: ITT, intention-to-treat; AT, as-treated; PP, per-protocol; IV, instrumental variable; PS, propensity score. 
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which might not be the same as how the children feh. For 
example, it was found that children with a congenital below- 
the-elbow deficiency reported better QoL than that perceived 
by their parents.^" Second, the early development of the CTN 
model of care was associated with suboptimal intervention 
fidelity. Implementation of the CTN was a major undertak- 
ing in changing systems and operations management. It took 
longer than anticipated for the network hosts to hire and 
train appropriate staff at the local team level, get local team 
sites operational, and engage all the needed agency partner 
staff in the teams. The delay in creating the child and family 
teams was the key challenge in organizing services for some 
children. About 52% of the children assigned to the CTN 
group did not receive CTN integrated care as planned during 
the follow-up period. The limited success in truly engaging 
and retaining families in the process of care is often the 
reason for failure to achieve the desired health outcome." 
Third, there was potential intervention contamination in the 
usual care group. Given that families within the same com- 
munity might be randomized to a different group, families 
in the usual care group might learn about the intervention 
from those assigned to the CTN group. The CTN trial was 
limited by insufficient resources and funding to employ a 
large-scale multicenter cluster design, which could protect 
against intervention contamination. Consequently, contami- 
nation of control children may have reduced the estimate of 
intervention effect and could have led to a type II error in 
the results.'* Fourth, our findings were limited by a relatively 
short period of observation after the initial launch of the inter- 
vention. For chronically ill children, the effect of integrated 
care on health outcomes would often accumulate over time. 
Thus, an interventional gain might be observed over a longer 
period of follow-up as practitioners became more proficient 
and team functioning became more collaborative. Fifth, our 
results were limited by missing data due to loss of patients 
to follow-up. We assumed that those data were missing at 
random and employed multiple imputation in the analysis. 
However, the assumption of data being missing at random 
is inherently untestable. Finally, the population targeted by 
the CTN may represent a subset of the defined CSHCN 
population accessing the services. Thus, our results may not 
be generalizable to a broader population of CSHCN, some 
of whom may not be accessing the services. The complex 
model of care and the heterogeneous population under study 
are the main challenges in assessing integrated health service 
interventions. '' 

Our findings should be interpreted in light of previously 
published evidence on the CTN intervention.^^-^' For the 



subgroup of children who received complete CTN integrated 
care, their QoL was influenced by complex interactions 
between team integration and parenting factors, and over- 
all, their QoL improved after 2 years.^^ In a cross-sectional 
analysis of the CTN children,^' we found that symptoms of 
psychiatric distress experienced by parents were associated 
with level of social support, family functioning, child behav- 
ior, and parenting style. All those factors could be mediated 
through coordinated and family-centered care. Also, the costs 
of total health and social services were higher in those parents 
with more symptoms of psychiatric distress. From an eco- 
nomic perspective, continuous and comprehensive services 
can save costs because giving people the services they need 
not only improves their QoL but also reduces use of other 
duplicative or otherwise fruitless services.'"' A recent sys- 
tematic review showed that coordinated and family-centered 
care was associated with improved outcomes for CSHCN."" 
A study by King et al''^ showed that services for children 
with neurodevelopmental disorders were most beneficial 
when they were delivered in a family-centered fashion and 
addressed parent-identified issues. The current evidence is 
primarily from nonrandomized studies. More randomized 
controlled trials are needed to assess the effect of integrative 
care that is continuous, comprehensive, coordinated, and 
family-centered for CSHCN. The CTN intervention is the first 
of its kind in Canada. It aligns with the advocacy of compre- 
hensive care for children's mental health,''^ family-centered 
rather than child-centered interventions,'** and partnerships 
with community psychology in the system of care.*' Con- 
sidering that 13%-18% of Canadian children had special 
health care needs, the CTN appears to be a milestone in the 
improvement of QoL for CSHCN through integrative care. 
At the same time, multiple community, family, parent, and 
child factors around the management of CSHCN make it 
challenging to evaluate this type of intervention. The ongoing 
implementation and evaluation of the CTN is an important 
step forward for Canadian community-based child care pro- 
grams targeting this vulnerable population. The methodologic 
limitations in our study inform further trials when evaluating 
similar interventions. 

Conclusion 

In this trial, we assessed the effect of CTN integrated care 
versus usual care on psychosocial QoL in CSHCN from 
Simcoe County and York Region over 2 years. We did not 
find a significant improvement in QoL for children in the 
CTN group compared with those in the usual care group. The 
value of continuous, comprehensive, coordinated, and family- 
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centered care for CSHCN is increasingly being recognized. 
At the same time, more research is needed to demonstrate the 
magnitude of the effect of integrated care on those children 
and their parents. Given that noncompliance is a common 
issue in randomized controlled trials, especially pragmatic 
trials, a better understanding of the performance of different 
analytical approaches is also crucial. Future simulation stud- 
ies are needed to provide insights into this question. 
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Supplementary materials 

Propensity score methods 

The propensity score (PS) is the probability of receiv- 
ing a treatment conditional on observed characteristics. 
Balancing on PS mimics randomization and produces 
comparable groups that are balanced on prognosis and 
observed confounding. We collected baseline variables that 
included child's age, sex, admission diagnosis, parent's age, 
marital status, education, distress, parenting style, family 
income, social support, and family functioning. Using those 
variables, we calculated the PS for receiving complete 
Children's Treatment Network (CTN) integrated care by a 
logistic regression model. Four PS methods were used to 
balance between treated and untreated children in the AT 
and PP analyses, ie, matching, stratification, weighting, and 
covariate adjustment. 

In PS matching, we created matched pairs of treated 
and untreated children by matching them within 0.2 of the 
standard deviation of the logit of the PS. We used a 1:1 
ratio to match the nearest children without replacement. 
This matching algorithm produced the least bias.' We then 
used the generalized estimating equation model to analyze 
the matched data. An exchangeable correlation structure 
was used. 

In PS stratification, children were divided into five equal 
strata by the quintiles of their PS.^ The outcome between 
treated and untreated children was compared directly in each 
stratum. The overall treatment effect is given by 



A.. 



where is the estimated treatment effect in stratum /. The 
variance of B „ is calculated by 

overall ■' 



where w. is the inverse of the estimated variance of B for 
stratum /. 

In the PS-weighted method, we balanced the children by 
the inverse probability of receiving CTN integrated care. The 
weight (w ) for each child can be calculated by 

w,. = — -h- , 

e,. 1-e,. 

where X is the treatment indicator and e is the estimated PS 
for child Confounding between the observed variables and 
treatment was eliminated in the weighted sample. We then 



directly compared the outcome between treated and untreated 
children using a weighted linear regression model. 

In the covariate adjustment method, we adjusted the PS 
as the sole covariate in the regression model for estimating 
the treatment effect. 

Calculating standard error for 
instrumental variable estimate 

Suppose we have two random variables, X and Y. A Taylor 
series expansion ofJ{x, y) about the values (x^, y^) is given by 



fix, y) = /(x„ , ^0 ) + (x - ) 



df(x,y) 



dx 



dnx,y) 



+(y-yo) 3 

dy 

f(Xo,yo) + ix-Xo) 

df(x,y) 



+ o{-) 



(^o.>'o) 



dx 



df{x,y) 



dy 



where o( ) represents the higher order terms which are omit- 
ted in this approximation. The instrumental variable (IV) 
estimator for the exposure X and the outcome 7 is given by 



Z->Y) 



where P^^_^.^ and P^2_^^ are the parameters of association 
between Z and Y and between Z and X, respectively. By the 
Taylor series expansion on /3y^ we have 



''(Z^X) 



(Z^Y) "^{Z^Y) 



"{Z^X) 



(Z^X) "(Z^X)J 



^(Z^Y) 



"(Z^X) 



where „ and 5,^ , are two values at which /J,,^ is dif- 

(Z— * / ) (Z— I V 

ferentiable. The variance of the IV estimator can then be 
approximated as 



1 , „ , \z^Y)^ 



"(Z^X) "{Z^X) 
'^^(Z^Y) 

hz^x? 



Under the assumption that P^^^^^ independent of 



the variance of the IV estimator is then 
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We substitute b^^^^ and b^^^,^ by the estimates of 
Piz^Y) and respectively, and approximate the 

^Var{J3^2^Y) ) ^iid ^F'flr(y^2^z) ) W the associated standard 
error. Thus, we can obtain an approximate variance of the 
IV estimate. In our analysis, we used the least squares esti- 
mate of /?,2^y) and associated standard error obtained from 
the linear regression model. For the exposure X (a binary 
indicator of whether or not a patient received complete CTN 
integrated care), the association between X and the IV can 
be calculated by 



P{X=\\Z = \)- P{X = 1 1 Z = 0) 



where P{X = 1|Z = 1) represents the proportion of treated 
patients in the CTN group; an6iP{X= 1|Z= 0) is always zero 



because the children in the usual care group are deemed to 
be untreated. The variance of P^^.^^^ is 

Var iP^^^^) = nP(X= 1|Z= 1) (1 - P(X= 1|Z= 1)) 

where n is the number of children in the CTN group. 



References 

1 . Austin PC. Optimal caliper widths for propensity-score matching when 
estimating differences in means and differences in proportions in obser- 
vational studies. Pharm Stat. 201 1;10(2):150-161. 

2. Rosenbaum PR, Rubin DB. Reducing bias in observational stud- 
ies using subclassification on the propensity score. J Am Stat Assoc. 
1984;79(387):5 16-524. 



Clinical Epidemiology 

Publish your work in this journal 

Clinical Epidemiology is an international, peer-reviewed, open access 
journal focusing on disease and drug epidemiology, identification of 
risk factors and screening procedures to develop optimal preventative 
initiatives and programs. Specific topics include: diagnosis, prognosis, 
treatment, screening, prevention, risk factor modification, systematic 

Submit your manuscript here: http://www.dovepress.com/clinical-epidemiology-journal 



Dovepress 



reviews, risk & safety of medical interventions, epidemiology & bio- 
statical methods, evaluation of guidelines, translational medicine, health 
policies & economic evaluations. The manuscript management system 
is completely online and includes a very quick and fair peer-review 
system, which is all easy to use. 



Clinical Epidemiology 2013:5 



submit your manuscript | www.dovepress.con i 

Dovepress 



385 



